Using Cognitive Models to Evaluate Ethnicity and Gender Differences

Differences in total test score for gender and ethnic subgroups are widely studied. The Attribute Hierarchy Method (AHM; Leighton & Gierl, 2007a), a diagnostic testing procedure, is used to evaluate differences for gender and ethnicity in the current study. is applied to gender, ethnicity, and gender by ethnicity comparisons for several cognitive models in Critical Reading and Mathematics. HCI values varied for the different ethnic and gender subgroups comparisons. This study enhances the evaluation of group performance and may assist with improving cognitive models. Educational testing has increased dramatically due to the No Child Left Behind (NCLB, 2001) legislation which requires each state to test students in Grades 3 through 8 in English/Language Arts (E/LA) and Mathematics. The NCLB mandate also requires states show 100% proficiency in E/LA and Mathematics by 2014 and to report growth Betebenner, 2002). In light of these requirements, diagnostic assessments are being used to assist with meeting these goals. Diagnostic assessments provide enhanced information required to improve student learning. Diagnostic assessments provide feedback to students and teachers about strengths and weaknesses of specific learning objectives. To ensure valid results for all students on these assessments, the question of test fairness must be addressed. Gender and ethnic test/item score differences are typically assessed using DIF is said to occur when the probabilistic differences in item scores occur after controlling for overall ability. DIF analyses are typically conducted for large-scale assessments. Unfortunately, there is no consensus on which DIF procedure works well for all student populations and it usually requires large sample sizes. In addition, many studies that have attempted to confirm, through content reviews, which items would indicated DIF and which group would be favored, have shown little success (e. There are several limitations to using DIF analysis in a diagnostic framework to identify test score differences: (1) information is gained mainly at the item level, (2) explanations about why the differences occur has been limited, (3) linking test performance to cognitive models has been lacking, and (4) most DIF analyses only focus on two groups. A similar method of confirming test fairness is needed in the context of diagnostic assessment. To address these limitations, we present a new method for examining group differences using a cognitive diagnostic assessment (CDA) framework, as evaluated using the attribute hierarchy method (AHM). The purpose of this study is to examine the differences in average Hierarchy Consistency Index (HCI; a …